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Method and device for visual masking of defects in matrix 
displays by using characteristics of the human vision system 

Technical field of the invention 

The present invention relates to a system and method for visually 
5 masking of pixel or sub-pixel defects present in matrix addressed electronic 
display devices, especially fixed format displays such as plasma displays, field 
emission displays, liquid crystal displays, electroluminescent (EL) displays, 
light emitting diode (LED) and organic light emitting diode (OLED) displays, 
especially flat panel displays used in projection or direct viewing concepts. 
10 The invention applies to both monochrome and colour displays and to 

emissive, transmissive, reflective and trans-reflective display technologies 
fulfilling the feature that each pixel or sub-pixel is individually addressable. 

Background of the invention 

At present, most matrix \ based display technologies are in its 

15 technological infancy compared to long established electronic image forming 
technologies such as Cathode Ray Tubes (CRT). As a result, many domains of 
image quality deficiency still exist and cause problems for the acceptance of 
these technologies in certain applications. 

Matrix based or matrix addressed displays are composed of individual 

20 image forming elements, called pixels (Picture Elements), that can be driven 
(or addressed) individually by proper driving electronics. The driving signals 
can switch a pixel to a first state, the on-state (at which luminance is emitted, 
transmitted or reflected), to a second state, the off-state (at which no 
luminance is emitted, transmitted or reflected) - see for example EP-1 17335 - 

25 or for some displays, one or any intermediate state between on or off 
(modulation of the amount of luminance emitted, transmitted or reflected) - see 
for example EP-0462619 and EP-1 17335. 

Since matrix addressed displays are typically composed of many 
millions of pixels, very often pixels exist that are stuck in a certain state (on, off 

30 or anything in between). Where pixel elements comprise multiple sub pixels, 
individually controllable or not, then one or more of the sub-pixel elements may 
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become stuck in a certain state. For example, a pixel structure may comprise 
three sub-pixel elements for red, green and blue colours respectively. If one of 
these sub-pixel elements becomes stuck in a certain state, then the pixel 
structure has a permanent colour shift. Mostly such problems are due to a 
5 malfunction in the driving electronics of the individual pixel (for instance a 
defect transistor). Other possible causes are problems with various production 
processes involved in the manufacturing of the displays, and/or by the physical 
construction of these displays, each of them being different depending on the 
type of technology of the electronic display under consideration. It is also 

10 possible that a pixel or sub-pixel element is not really stuck in a state, but 
shows a luminance or colour behaviour that is significantly different from the 
pixels or sub-pixels in its neighbourhood. For instance, but not limited to: a 
defective pixel shows a luminance behaviour that differs more than 20% (at 
one or more video levels) from the pixels in its neighbourhood, or a defective 

15 pixel shows a dynamic range (maximum luminance / minimum luminance) that 
differs more than 15% from the dynamic range of pixels in its neighbourhood, 
or a defective pixel shows a colour shift greater than a certain value comparing 
to an average or desired value for the display. Of course other rules are 
possible to determine whether a pixel or sub-pixel is defective or not (any 

20 condition that has a potential danger for image misinterpretation can be 
expressed in a rule to determine whether a pixel is a defective pixel). Bright or 
dark spots due to dust for example may also be considered as pixel defects. 
The exact reason for the defective pixel is not important for the present 
invention. 

25 Defective pixels or sub-pixels are typically very visible for the user of the 

display. They result in a significantly lower (subjective) image quality, can be 
very annoying or disturbing for the display-user and for demanding 
applications (such as medical imaging, in particular mammography) the 
defective pixels or sub-pixels can even make the display unusable for the 

30 intended application, as it can also result in wrong interpretation of the image 
being displayed. For applications where image fidelity is required to be high, 
such as for example in medical applications, this situation is unacceptable. 
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US-5,504,504 describes a method and display system for reducing the 
visual impact of defects present in an image display. The display includes an 
array of pixels, each non-defective pixel being selectively operable in response 
to input data by addressing facilities between an u on M state, whereat light is 
5 directed onto a viewing surface, and an "off state, whereat light is not directed 
onto the viewing surface. Each defective pixel is immediately surrounded by a 
first ring of compensation pixels adjacent to the central defective pixel. The 
compensation pixels are immediately surrounded by a second ring of 
reference pixels spaced from the central defective pixel. The addressing 

10 circuit-determined value of at least one compensation pixel in the first ring 
surrounding the defective pixel is changed from its desired or intended value to 
a corrective value, in order to reduce the visual impact of the defect. In one 
embodiment, the value of the compensation pixels is selected such that the 
average visually defected value for all of the compensation pixels and the 

15 defective pixel is equal to the intended value of the defective pixel. In another 
embodiment, the values of the compensation pixels are adjusted by adding an 
offset to the desired value of each compensation pixel. The offset is chosen 
such that the sum of the offset values is equal to the intended value of the 
defective pixel. 

20 It is a disadvantage of the solution proposed in the above document that 

a trial and error method is required for every other display in order to obtain a 
reasonable correction result. 

From WO 03/100756 it is known to mask a faulty pixel having a defect 
sub-pixel for a display system with pixels having a set of primary sub-pixels 

25 with an additional redundant sub-pixel. The masking is performed by reducing 
an error between a desired perceptive characteristic of said faulty pixel and 
modified perceptive characteristics of said pixel. In other words, the method is 
focussed on obtaining a desired perceptive characteristic for the faulty pixel, 
whereby the use of a redundant sub-pixel is required. It is a disadvantage of 

30 the method of the above document that a redundant sub-pixel is necessary for 
each and every pixel. The document does not describe how to mask defects in 
a display system without additional redundant pixel. 
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Summary of the invention 

It is an object of the present invention to provide a method and device 
for making pixel defects less visible and thus avoid wrong image interpretation, 
the method being usable for different types of matrix displays without a trial 
5 and error method being required to obtain acceptable correction results. 

The above objective is accomplished by a method and device according 
to the present invention. 

In a first aspect, the present invention provides a method for reducing 
the visual impact of defects present in a matrix display comprising a plurality of 
10 display elements, the method comprising: 

providing a representation of a human vision system, 

characterising at least one defect present in the display, the defect being 

surrounded by a plurality of non-defective display elements, 

deriving drive signals for at least some of the plurality of non-defective display 

15 elements in accordance with the representation of the human vision system 
and the characterising of the at least one defect, to thereby minimise an 
expected response of the human vision system to the defect, and 
driving at least some of the plurality of non-defective display elements with the 
derived drive signals. In a further aspect, the present invention provides a 

20 method for reducing the visual impact of defects present in a matrix display 
comprising a plurality of pixels, the pixels comprising at least three sub-pixels, 
each sub-pixel intended for generating a sub-pixel colour which cannot be 
obtained by a linear combination of the sub-pixel colours of the other sub- 
pixels of the pixel, the method comprising: 

25 providing a representation of a human vision system, 

characterising at least one defect sub-pixel present in the display, the defect 
sub-pixel intended for generating a first sub-pixel colour and being surrounded 
by a plurality of non-defective sub-pixels, 

deriving drive signals for at least some of the plurality of non-defective sub- 
30 pixels in accordance with the representation of the human vision system and 
the characterising of the at least one defect sub-pixel, to thereby minimise an 
expected response of the human vision system to the defect sub-pixel, and 
driving at least some of the plurality of non-defective sub-pixels with the 
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derived drive signals, wherein minimising the response of the human vision 
system to the defect sub-pixel comprises changing the light output value of at 
least one non-defective sub-pixel for generating another sub-pixel colour, said 
another sub-pixel colour differing from said first sub-pixel colour. 
5 Minimising the response of the human vision system to the defect sub- 

pixel may comprise introducing a light output deviation in at least one non- 
defective sub-pixel being part of the same pixel as said defect sub-pixel. The 
light output deviation of the defect sub-pixel thereby is defined as the 
difference in light output between the defect sub-pixel and the light output of 

10 the same sub-pixel or a similar sub-pixel having the same properties, in a non- 
defect state. Said introduced light output deviation may be similar to the light 
output deviation caused by the defect sub-pixel. This means that the light 
output deviation of the defect sub-pixel and the introduced light output 
deviation of the non-defective sub-pixel differ 50% or less, preferably 20% or 

15 less, more preferred 10% or less, and still more preferred are equal or 
substantially equal. 

Alternatively said light output deviation may be such that a total light 
output of said pixel is substantially equal to a total light output of a pixel having 
no defect sub-pixels. This means that the total light output of a pixel having no 

20 defect sub-pixels, and the total light output of the same pixel having a defect 
sub-pixel which is corrected for according to the present invention, differ 50% 
or less, preferably 20% or less, more preferred 10% or less, and still more 
preferred are equal. 

Deriving drive signals for at least some of the plurality of non-defective 

25 sub-pixels furthermore may be performed by incorporating a correction for at 
least one of a distance between said human vision system and said display, a 
viewing angle between said human vision system and said display and a 
presence of environmental stray light. 

Characterising at least one defect sub-pixel present in the display may 

30 comprise storing characterisation data characterising the location and non- 
linear light output response of individual sub-pixels, the characterisation data 
representing light outputs of an individual sub-pixels as a function of its drive 
signals. 
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A method according to the present invention may further comprise 
generating the characterisation data from images captured from sub-pixels. 
Generating the characterisation data may comprise building a display element 
profile map representing characterisation data for each sub-pixel of the 
5 display. 

Providing a representation of the human vision system may comprise 
calculating an expected response of a human eye to a stimulus applied to a 
sub-pixel. For calculating the expected response of a human eye to a stimulus 
applied to a sub-pixel, use may be made of any of a point spread function, a 
10 pupil function, a line spread function, an optical transfer function, a modulation 
transfer function or a phase transfer function of the eye. These functions may 
be described analytically, for example based on using any of Tailor, Seidel or 
Zernike polynomials, or numerically. 

In a method according to the present invention, when minimising the 
15 response of the human vision system to the defect sub-pixel, boundary 
conditions may be taken into account. 

Minimising the response of the human vision system may be carried out 
in real-time or off-line. 

A defect may be caused by a defective sub-pixel or by an external 
20 cause, such as dust adhering on or between sub-pixels for example. 

In a second aspect, the present invention provides a system for 
reducing the visual impact of defects present in a matrix display comprising a 
plurality of display elements and intended to be looked at by a human vision 
system, first characterisation data for a human vision system being provided, 
25 the system comprising: 

a defect characterising device for generating second characterisation data for 
at least one defect present in the display, the defect being surrounded by a 
plurality of non-defective display elements, 

a correction device for deriving drive signals for at least some of the plurality of 
30 non-defective display elements in accordance with the first characterisation 
data and the second characterising data, to thereby minimise an expected 
response of the human vision system to the defect, and 
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means for driving at least some of the plurality of non-defective display 
elements with the derived drive signals. 

In a further aspect, the present invention provides a system for reducing 
the visual impact of defects present in a matrix display comprising a plurality of 
5 pixels, said pixels comprising at least three sub-pixels, each sub-pixel intended 
for generating a sub-pixel colour which cannot be obtained by a linear 
combination of the sub-pixel colours of the other sub-pixels of the pixel, and 
intended to be looked at by a human vision system, first characterisation data 
for a human vision system being provided, the system comprising: 
10 a defect characterising device for generating second characterisation data for 
at least one defect sub-pixel present in the display, the defect sub-pixel 
intended for generating a first sub-pixel colour and being surrounded by a 
plurality of non-defective sub-pixels, 

a correction device for deriving drive signals for at least some of the plurality of 
1 5 non-defective sub-pixels in accordance with the first characterisation data and 
the second characterising data, to thereby minimise an expected response of 
the human vision system to the defect sub-pixel, and 

means for driving at least some of the plurality of non-defective sub-pixels with 
the derived drive signals, wherein the correction device comprises means to 

20 change the light output value of at least one non-defective sub pixel intended 
for generating another sub-pixel colour, said another sub-pixel colour differing 
from said first sub-pixel colour. 

The correction device may comprise means for introducing a light output 
deviation in at least one non-defective sub-pixel being part of the same pixel 

25 as said defect sub-pixel. Said light output deviation may be similar to a light 
output deviation caused by the defect sub-pixel. The light output deviation of 
the defect sub-pixel thereby is defined as the difference in light output between 
the defect sub-pixel and the light output of the same sub-pixel or a similar sub- 
pixel having the same properties, in a non-defect state. According to 

30 embodiments of the present invention, the light output deviation of the defect 
sub-pixel and the introduced light output deviation of the non-defective sub- 
pixel differ 50% or less, preferably 20% or less, more preferred 10% or less, 
and still more preferred are equal or substantially equal. 



WO 2005/052902 PCT/EP2004/0 13436 

8 

Alternatively said light output deviation is such that a light output of said 
pixel is substantially equal to a light output of a pixel having no defect sub- 
pixels. This means that the total light output of a pixel having no defect sub- 
pixels, and the total light output of the same pixel having a defect sub-pixel 
5 which is corrected for according to the present invention, differ 50% or less, 
preferably 20% or less, more preferred 10% or less, and still more preferred 
are equal. 

The correction device for deriving driving signals may be adapted for 
deriving driving signals incorporating a correction for at least one of a distance 

10 between said human vision system and said display, a viewing angle between 
said human vision system and said display and a presence of environmental 
stray light.The defect sub-pixel characterising device may comprise an image 
capturing device for generating an image of the sub-pixels of the display. The 
defect sub-pixel characterising device may also comprise a sub-pixellocation 

15 identifying device for identifying the actual location of individual sub-pixels of 
the display. 

In a system according to the present invention, for providing the first 

characterisation data, a vision characterising device having calculating means 

for calculating the response of a human eye to a stimulus applied to a sub- 
20 pixel may be provided. 

In a third aspect, the present invention provides a matrix display device 

for displaying an image intended to be looked at by a human vision system, 

the matrix display device comprising: 

a plurality of display elements, 
25 a first memory for storing first characterisation data for a human vision system, 

a second memory for storing second characterisation data for at least one 

defect present in the display device, 

a modulation device for modulating, in accordance with the first 
characterisation data and the second characterisation data, drive signals for 
30 non-defective display elements surrounding the defect so as to reduce the 
visual impact of the defect present in the matrix display device. 

In a further aspect, the present invention provides a matrix display 
device for displaying an image intended to be looked at by a human vision 
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system, the matrix display device comprising: 

a plurality of pixels, said pixels comprising at least three sub-pixels, each sub- 
pixel intended for generating a sub-pixel colour which cannot be obtained by a 
linear combination of the sub-pixel colours of the other sub-pixels of the pixel, 
5 a first memory for storing first characterisation data for a human vision system, 
a second memory for storing second characterisation data for at least one 
defect sub-pixel present in the display device, the defect sub-pixel intended for 
generating a first sub-pixel colour, 

a modulation device for modulating, in accordance with the first 
10 characterisation data and the second characterisation data, drive signals for 
non-defective sub-pixels surrounding the defect sub-pixel so as to reduce the 
visual impact of the defect sub-pixel present in the matrix display device, 
wherein modulating drive signals comprises changing the light output value of 
at least one non-defective sub-pixel intended for generating another sub-pixel 
15 colour, said another sub-pixel colour differing from said first sub-pixel colour. 

The first and the second memory may physically be a same memory 
device. 

In a fourth aspect, the present invention provides a control unit for use 
with a system for reducing the visual impact of defects present in a matrix 
20 display comprising a plurality of display elements and intended to be looked at 
by a human vision system, the control unit comprising: 

a first memory for storing first characterisation data for a human vision system, 
a second memory for storing second characterisation data for at least one 
defect present in the display, and 
25 modulating means for modulating, in accordance with the first characterisation 
data and the second characterisation data, drive signals for non-defective 
display elements surrounding the defect so as to reduce the visual impact of 
the defect. 

In a further aspect, the present invention provides a control unit for use 
30 with a system for reducing the visual impact of defects present in a matrix 
display comprising a plurality of pixels, said pixels comprising at least three 
sub-pixels, each sub-pixel intended for generating a sub-pixel colour which 
cannot be obtained by a linear combination of the sub-pixel colours of the 
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other sub-pixels of the pixel, and intended to be looked at by a human vision 
system, the control unit comprising: 

a first memory for storing first characterisation data for a human vision system 
a second memory for storing second characterisation data for at least one 
5 defect sub-pixel present in the display, the defect sub-pixel intended for 
generating a first sub-pixel colour and 

modulating means for modulating, in accordance with the first characterisation 
data and the second characterisation data, drive signals for non-defective sub- 
pixels surrounding the defect sub-pixel so as to reduce the visual impact of the 

10 defect sub-pixel, wherein modulating drive signals comprises changing the 
light output value of at least one non-defective sub-pixel intended for 
generating another sub-pixel colour, said another sub-pixel colour differing 
from said first sub-pixel colour. 

The present invention thus solves the problem of defective pixels and/or 

15 sub-pixels in matrix displays by making them almost invisible for the human 
eye under normal usage circumstances. This is done by changing the drive 
signal of non-defective pixels and/or sub-pixels in the neighbourhood of the 
defective pixel or sub-pixel. 

In the following description the pixels or sub-pixels that are used to 

20 mask the defective pixel are called "masking elements" and the defective pixel 
or sub-pixel itself is called "the defect" . 

By a defective pixel or sub-pixel is meant a pixel that always shows the 
same luminance, i.e. a pixel or sub-pixel stuck in a specific state (for instance, 
but not limited to, always black, or always full white) and/or colour behaviour 

25 independent of the drive stimulus applied to it, or a pixel or sub-pixel that 
shows a luminance or colour behaviour that shows a severe distortion 
compared to non-defective pixels or sub-pixels of the display. For example a 
pixel that reacts to an applied drive signal, but that has a luminance behaviour 
that is very different from the luminance behaviour of neighbouring pixels, for 

30 instance significantly more dark or bright than surrounding pixels, can be 
considered a defective pixel. 

By visually masking is meant minimising the visibility and negative 
effects of the defect for the user of the display. 
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The present invention discloses a mathematical model that is able to 
calculate the optimal driving signal for the masking elements in order to 
minimise the visibility of the defect(s). The same algorithm can be used for 
every display configuration because it uses some parameters that describe the 
5 display characteristics. A mathematical model based on the characteristics of 
the human eye is used to calculate the optimal drive signals of the masking 
elements. The model describes algorithms to calculate the actual response of 
the human eye to the superposition of the stimulus applied (in casu to the 
defect and to the masking pixels). In this way the optimal drive signals of the 
0 masking elements can be described as a mathematical minimisation problem 
of a function with one or more variables. It is possible to add one or more 
boundary conditions to this minimisation problem. Examples when extra 
boundary conditions are needed are in case of defects of one or more masking 
elements, limitations to the possible drive signal of the masking elements, 
5 dependencies in the drive signals of masking elements ... 

The present invention cannot repair the defective pixels but makes the 
defects (nearly) invisible and thus avoids wrong image interpretation. 

The above and other characteristics, features and advantages of the 
present invention will become apparent from the following detailed description, 
20 taken in conjunction with the accompanying drawings, which illustrate, by way 
of example, the principles of the invention. This description is given for the 
sake of example only, without limiting the scope of the invention. The 
reference figures quoted below refer to the attached drawings. 

Brief description of the drawings 

25 Fig. 1a illustrates a matrix display having greyscale pixels with equal 

luminance, and Fig. 1b illustrates a matrix display having greyscale pixels with 
unequal luminance. 

Fig. 2a illustrates an LCD display having an RGB-stripe pixel 
arrangement: one pixel comprises three coloured sub-pixels in stripe ordering, 

30 and the display has a defective green sub-pixel that is always fully on, and a 
defective red sub-pixel that is always off. Fig. 2b illustrates a greyscale LCD 
based matrix display having unequal luminance in sub-pixels. 
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Fig. 3a illustrates an analytical point spread function (PSF) in case the 
optics is considered to be diffraction-limited only; Fig. 3b and Fig. 3c illustrate 
numerical PSFs that are measured on test subjects. 

Fig. 4a shows the eye response to a single pixel defect in the image 
5 plane if no masking is applied. Fig. 4b shows the eye response to the same 
pixel defect but after masking with 24 masking pixels has been applied. Fig. 4c 
shows the centre locations of the PSFs in the image plane of the masking 
pixels and the pixel defect. 

Fig. 5a illustrates nine pixels each having three sub-pixels and two 
10 domains. Fig. 5b shows one of such pixels in detail. 

Fig. 6 illustrates the transformation from a driving level to a luminance 

level. 

Fig. 7a shows a real green sub-pixel defect present in a display, and 
Fig. 7b shows the same green sub-pixel defect and artificial red and blue sub- 
1 5 pixel defects introduced to retain a colour co-ordinate of the pixel which is as 
close to the correct colour co-ordinate as possible. 

Fig. 8 illustrates possible locations for a real-time correction system 
according to any embodiment of the present invention. 

In the different figures, the same reference signs refer to the same or 
20 analogous elements. 

Description of illustrative embodiments 

The present invention will be described with respect to particular 
embodiments and with reference to certain drawings but the invention is not 
limited thereto but only by the claims. The drawings described are only 
25 schematic and are non-limiting. In the drawings, the size of some of the 
elements may be exaggerated and notdrawn on scale for illustrative purposes. 
Where the term "comprising" is used in the present description and claims, it 
does not exclude other elements or steps. 

In the present description, the terms "horizontal" and "vertical" are used 
30 to provide a co-ordinate system and for ease of explanation only. They refer to 
a co-ordinate system with two orthogonal directions which are conveniently 
referred to as vertical and horizontal directions. They do not need to, but may, 
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refer to an actual physical direction of the device. In particular, horizontal and 
vertical are equivalent and interchangeable by means of a simple rotation 
through and odd multiple of 90°. 

A matrix addressed display comprises individual display elements. The 

5 display elements, either themselves or in groupings, are individually 
addressable to thereby display or project an arbitrary image. In the present 
description, the term "display elements" is to be understood to comprise any 
form of element which modulates a light output, e.g. elements which emit light 
or through which light is passed or from which light is reflected. The term 

10 "display" includes a projector. A display element may therefore be an 
individually addressable element of an emissive, transmissive, reflective or 
trans-reflective display, especially a fixed format display. The term "fixed 
format" relates to the fact that an area of any image to be displayed or 
projected is associated with a certain portion of the display or projector, e.g. in 

15 a one-to-one relationship. Display elements may be pixels, e.g. in a greyscale 
LCD, as well as sub-pixels, a plurality of sub-pixels forming one pixel. For 
example three sub-pixels with a different colour, such as a red sub-pixel, a 
green sub-pixel and a blue sub-pixel, may together from one pixel in a colour 
display such as an LCD. Whenever the word pixel is used, it is to be 

20 understood that the same may hold for sub-pixels, unless the contrary is 
explicitly mentioned. 

The invention will be described with reference to flat panel displays but 
is not limited thereto. It is understood that a flat panel display does not have to 
be exactly flat but includes shaped or bent panels. A flat panel display differs 

25 from a display such as a cathode ray tube in that it comprises a matrix or array 
of "cells" or "pixels" each producing or controlling light over a small area. 
Arrays of this kind are called fixed format arrays. There is a relationship 
between the pixel of an image to be displayed and a cell of the display. Usually 
this is a one-to-one relationship. Each cell may be addressed and driven 

30 separately. It is not considered a limitation on the present invention whether 
the flat panel displays are active or passive matrix devices. The array of cells is 
usually in rows and columns but the present invention is not limited thereto but 
may include any arrangement, e.g. polar or hexagonal. The invention will 
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mainly be described with respect to liquid crystal displays but the present 
invention is more widely applicable to flat panel displays of different types, 
such as plasma displays, field emission displays, EL-displays, OLED displays 
etc. In particular the present invention relates not only to displays having an 
5 array of light emitting elements but also displays having arrays of light emitting 
devices, whereby each device is made up of a number of individual elements. 
The displays may be emissive, transmissive, reflective, or trans-reflective 
displays. 

Further the method of addressing and driving the pixel elements of an 

10 array is not considered a limitation on the invention. Typically, each pixel 
element is addressed by means of wiring but other methods are known and 
are useful with the invention, e.g. plasma discharge addressing (as disclosed 
in US-6,089,739) or CRT addressing. 

A matrix addressed display 12 comprises individual pixels 14. These 

15 pixels 14 can take all kinds of shapes, e.g. they can take the forms of 
characters. The examples of matrix displays 12 given in Fig. 1a to Fig. 2b have 
rectangular or square pixels 14 arranged in horizontal rows and vertical 
columns. Fig. 1a illustrates an image of a perfect display 12 having equal 
luminance response in all pixels 14 when equally driven. Every pixel 14 driven 

20 with the same signal renders the same luminance. In contrast, Fig. 1b 
illustrates an image of a display 12 where the pixels 14 of the display 12 are 
also driven by equal signals, but where the pixels 14 render a different 
luminance, as can be seen by the different grey values. Pixel 16 in the display 
12 of Fig. 1b is a defective pixel. Fig. 1b shows a monochrome pixel structure 

25 with one defective pixel 16 that is always in an intermediate pixel state. 

Fig. 2a shows a typical RGB-stripe pixel arrangement of a colour LCD 
display 12: one pixel 14 consists of three coloured sub-pixels 20, 21, 22 in 
stripe ordering. These three sub-pixels 20, 21, 22 are driven individually to 
generate colour images. In Fig. 2a there are two defective sub-pixels present: 

30 a defective red sub-pixel 24 that is always off and a defective green sub-pixel 

25 that is always fully on. 

Fig. 2b shows an asymmetric pixel structure that is often used for high- 
resolution monochrome displays. In Fig. 2b, one monochrome pixel 14 
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consists of three monochrome sub-pixels. Depending on the panel type and 
driving electronics the three sub-pixels of one pixel are driven as a unit or 
individually. Fig. 2b shows 3 pixel defects: a complete defective pixel 16 in 
"always on" state and two defective sub-pixels 27, 28 in "always off state that 

5 happen to be located in a same pixel 1 4. 

The spatial distribution of the luminance differences of the pixels 14 can 
be arbitrary. It is also found that with many technologies, this distribution 
changes as function of the applied drive to the pixels indicating different 
response relationships for the pixels 14. For a low drive signal leading to low 

10 luminance, the spatial distribution pattern can differ from the pattern at higher 
driving signal. 

The optical system of the eye, in particular of the human eye, comprises 
three main components: the cornea, the iris and the lens. The cornea is the 
transparent outer surface of the eye. The pupil limits the amount of light that 

1 5 reaches the retina and it changes the numerical aperture of the optical system 
of the eye. By applying tension to the lens, the eye is able to focus on both 
nearby and far away objects. The optical system of the eye is very complex but 
the process of image formation can be simplified by using a "black-box" 
approach. The behaviour of the black box can be described by the complex 

20 pupil function: 

P(x,y) • exp[-i(27r/A) • W(x,y)]. 
In this formula i stands for V-1 and A is the wavelength of the light. The pupil 
function consists of two parts: the amplitude component P(x,y) which defines 
the shape, size and transmission of the black box; and the wave aberration 
25 W(x,y) which defines how the phase of the light has changed after passing 

through the black box. 

Once the nature of the light (that passed through the black box, in this 
case the eye) is known, the image formation process can be described by the 
point spread function (PSF). The PSF describes the image of a point source 
30 formed by the black box. Most lenses, including the human lens, are not 
perfect optical systems. As a result when visual stimuli are passed through the 
cornea and lens the stimuli undergo a certain degree of degradation or 
distortion. This degradation or distortion can be represented by projecting an 
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exceedingly small dot of light, a point, through a lens. The image of this point 
will not be the same as the original because the lens will introduce a small 
amount of blur. 

The PSF of the eye can be calculated using the Fraunhofer 
5 approximation: 

PSF(x',y')= K 1 FT{P(x,y) exp[-i(27r/A) W(x,y)]}| 2 
where FT stands for the two-dimensional Fourier transform, usually denoted as 
F(x\y')= FT{f(x,y)}, and K is a constant. The | j represents the modulus- 
operator. In case of the human eye, the PSF describes the image of a point 

1 0 source on the retina. To describe a complete object one can think of an object 
as a combination or a matrix of (a potentially exceedingly large number or 
infinite number of) point sources. Each of these point sources is then projected 
on the retina as described by the same PSF (this approximation is strictly only 
valid if the object is small and composed of a single wavelength). 

1 5 Mathematically this can be described by means of a convolution: 

l(x',y')= PSF ® O(x'.y') 
where l(x',y') is the resulting image on the retina, PSF the point spread function 
and 0(x',y') the object representation at the image-plane. Typically this 
convolution will be computed in the Fourier domain by multiplying the Fourier 

20 transforms of both the PSF and the object and then applying the inverse 

Fourier transform to the result. 

It is common practice in vision applications to describe the wave 
aberration W(x,y) mathematically by means of a set of polynomials. Often 
Seidel polynomials are used, but also Taylor polynomials and Zernike 
25 polynomials are common choices. Especially Zernike polynomials have 
interesting properties that make wave aberration analysis much easier. Often 
unknown wave aberrations are approximated by Zernike polynomials; the 
coefficients of the polynomials are typically determined by performing a least- 
square fit. 

30 For the present invention, it is not considered a limitation on the 

invention how the complex pupil function or the PSF is described. This can be 
done analytically (for instance but not limited to a mathematical function in 
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Cartesian or polar co-ordinates, by means of standard polynomials, or by 
means of any other suitable analytical method) or numerically by describing 
the function value at certain points. It is also possible to use (instead of the 
PSF) other (equivalent) representations of the optical system such as but not 
5 limited to the Pupil Function (or aberration)', the 'Line Spread Function (LSF)\ 
the 'Optical Transfer Function (OTF)', the 'Modulation Transfer function (MTF)' 
and 'Phase Transfer Function (PTF)\ Clear mathematical relations exist 
between all these representation-methods so that it is possible to transform 
one form into another form. Fig. 3a shows an analytical PSF in case the optics 

10 is considered to be diffraction-limited only. It is to be noted that the PSF is 
clearly not a single point, i.e. the image of a point source is not a point, the 
central zone of the diffraction-limited PSF is called an airy disc. Fig. 3b and 
Fig. 3c show (numerical) PSFs that were measured on test subjects. Here 
again it can be seen that the PSF is not a point. 

15 As the PSF of each optical system may be different, correction 

according to the present invention can be made user specific by using eye 
characteristics, and thus PSFs, which are specific for that user. 

Based on the PSF of the optical system, according to an aspect of the 
present invention, the response or expected response of the eye to a defective 

20 pixel can be mathematically described. Therefore the defective pixel is treated 
as a point source with an "error luminance" value dependent on the defect 
itself and the image data that should be displayed at the defect location at that 
time. For instance if the defective pixel is driven to have luminance value 23 
but due to the defect it outputs luminance value 3, then this defect is treated as 

25 a point source with error luminance value -20. It is to be noted that this error 
luminance value can have both a positive and a negative value. Supposing 
that some time later this same defective pixel is driven to show luminance 
value 1 but due to the defect it still shows luminance value 3, then this same 
defective pixel will be treated as a point source with error luminance value +2. 

30 As described above, this point source with a specific error luminance 

value will result in a response of the eye as described by the PSF. Because 
this response is typically not a single point, it is possible to use pixels and/or 
sub pixels in the neighbourhood of the defective pixel to provide some image 
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improvement. These neighbouring pixels are called masking pixels and can be 
driven in such a way as to minimise the response of the eye to the defective 
pixel. According to the present invention, this is achieved by changing the drive 
signal of the masking pixels such that the superposition of the image of the 
masking pixels and the image of the defective pixel results in a lower or 
minimal response of the human eye. Mathematically this can be expressed as 
follows: 



[Cj , C 2 , . . . , C n ] — min cl cl cn 4 



+CO+Q0 



J J cost/unction 



-to- 00 



C v PSF{x , -xV,y-yY)+ 

C 2 J>SF{x'-x2' , y'-y2') + . . . + 

C„ .PSF(x'-xn' , y'-yri) + 

Ej>SF(x\y\x\y 



cbc'dy' 



(Eq. 1) 

10 where C1, .... Cn are the luminance values that have to be superposed to the 
masking pixels M1, .... Mn with relative locations (x1, y1), (x2, y2), (xn, yn) 
in order to obtain minimal eye response to the defect. The function 
costfunction(v, x\ y') is calculates a "penalty" value from the eye response at 
location (x'.y"). Some examples (not limited to) are costfunction(v, x\ y')= v 2 , 

15 costfunction(v, x\ y')= abs(v), costfunction(v, x', y')= v 2 /(sqrt(x' 2 +y' 2 )). It is to be 
noted that the Cartesian coordinate system (x'.y') (with accents) is defined in 
the image plane on the retina with origin being the centre of the PSF(x',y') of 
the defect. The Cartesian co-ordinate system (x,y) is defined in the object 
plane of the display where (x,y) denotes the location of the masking pixels 

20 relative to the defect. The relation between these two co-ordinate systems can 
be expressed as (x", y')= (C*x, C*y) where C is a constant that defines the 
magnification in the image plane (depends on, among others, the object 
distance). Fig. 4a shows the eye response to a single defective pixel in the 
image plane if no masking is applied. Tig. 4b shows the eye response to the 

25 same defective pixel but after masking using 24 masking pixels (neighbours of 
the defective pixel) has been applied. Fig. 4c shows the centre locations of the 
PSFs in the image plane of the masking pixels and the defective pixel (central 
point). These simulations have been performed with the diffraction limited PSF 
and the minimisation was done numerically by using a least square error 

30 method. 
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The present invention is not limited to any particular co-ordinate system 
such as the Cartesian co-ordinate system as used above; other systems are 
also possible, for instance, but not limited to, a polar co-ordinate system. 

According to the present invention, the problem of finding an optimal 
5 correction luminance of the masking pixels is translated into a well-understood 
minimisation problem. It is to be noted that this mathematical description is 
very general: it does not impose any limitation on the number of masking pixels 
nor on the location of these masking pixels. The pixels also do not need to be 
located in any particular pixel structure: the algorithm can handle all possible 
10 pixel organisations. Also the defect itself is not necessarily located at a pixel 
location: for example some dust between two pixels can cause a permanent 
bright spot. 

The algorithm above describes a general method to calculate optimal 
driving signals for masking pixels in order to minimise the eye response to the 
1 5 defect. 

In practice, however, some special situations exist that may require 
additions to the described algorithm. 

A first special situation is when the pixels cannot be driven individually, 
but are rather driven in groups. High-resolution monochrome LCDs, for 

20 example, often have a pixel structure where one monochrome pixel consists of 
three monochrome sub-pixels that are equally and simultaneously driven, as 
illustrated in Fig. 2b. In such a situation a boundary condition needs to be 
applied to the minimisation problem to be solved, in order to respect this 
driving method. In the case of three equally and simultaneously driven sub- 

25 pixels, the boundary condition should state that the correction coefficients of 

♦ 

each of the simultaneously driven sub-pixels within a same pixel should have a 
same value. 

A second special situation occurs when pixels have a limited driving 
range. It is possible that the above-described correction algorithm would result 
30 in a required luminance value for a masking pixel that lies outside of the 
luminance range of the pixel. Introducing a boundary condition that limits the 
driving value of all pixels solves this problem. Such type of boundary condition 
can be stated as: 
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LL <= Pixel value + correction value <= UL 
and this for all masking pixels. In this expression LL is the lower driving limit of 
the pixels and UL is the upper driving limit. "Pixel value" is the normal 
(uncorrected) pixel value of the pixel and "correction value" is the calculated 
correction value for that masking pixel. 

Furthermore, the requirement that the final driving value of the masking 
pixel should be an integer can be a boundary condition to be used. 

A third special situation occurs when there are multiple defects in a 
small area, the small area being the area that contains all masking pixels for 
one particular defect. In this case it might not be possible to assign the 
required value to all masking pixels. In this case the mathematical description 
should be restated: one of the defects should be chosen as the centre of both 
the image plane and object plane co-ordinate systems. Then the algorithm 
should minimise the total response to all the defects and all used masking 
pixels in this area as shown in the formula below: 

(CI FSFfr-xV , y x -yV) + . . . 



[Cl,C2,...,Cw] = min cl _ Cw < 



-f oof oo 



J J zostfunction 



— CO— 00 
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+ CnJ > SF{x x -xn\y-yn ; ) 
+ E\.PSF(x\y) 
+ E2.PSF{x*-ex2 x , ?-ey2) 
... + EmJPSF(x'-exm\ y'-eynt) 



dx'dy' 



where C1, Cn are the correction values to be superposed to the masking 
pixels and E1, Em are the error luminance values of the defects in the 
neighbourhood. It is to be noted that in this case defect 1 was chosen as 
0 origin. 

A fourth special situation occurs when pixels (or defects) are larger so 
that they cannot be modelled anymore by a point source. To solve this, the 
defect should be modelled as a (possibly infinite) number of point sources. An 
example could be a dual domain in-plane switching (IPS) LCD panel where 
5 pixels consist of two domains. Such pixels can be modelled by two or more 
point sources that do not have necessarily the same luminance value. Fig. 5a 
shows nine pixels 50 each having three sub-pixels 51 and each sub-pixel 51 
having two domains 52, 53. Fig. 5b shows one pixel 50 in detail. In this 
situation it could be necessary to treat each pixel 50 as a superposition of 6 
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point sources. Because the pixel 50 can only be driven as a unit, a boundary 
condition is required stating that the 6 correction coefficients of each pixel 50 
should be equal. 

The algorithms described use luminance values and not driving values. 
5 Typical displays however have no linear relation between driving level of a 
pixel and resulting luminance value. Therefore, in a realistic display system, 
the calculated luminance correction should be transformed into a required 
drive level correction. Typically a display system has one or more look-up 
tables (LUTs) connected to a panel with a specific gamma curve. The 

10 conversion from luminance value to driving value is straightforward by applying 
the inverse operations. It is to be noted that depending on the exact location 
where the correction will be applied, the LUT inversion may or may not be 
necessary. Fig. 6 shows a typical transformation from driving level to the 
resulting luminance level. 

15 The above embodiments of the present invention all relate to 

monochrome displays. In case of colour displays there are three possibilities to 
calculate the correction. 

A first method is to use only masking sub-pixels of the same colour as 
the defective sub-pixel. This method is simple, but can introduce visible colour 

20 shifts since the colour value of the defective pixel and the masking pixels can 
change. 

Therefore, a second method is proposed, according to which artificial 
defects are introduced such that the colour points or colour co-ordinates of the 
defective pixel and the masking pixels change only a little or do not change at 

25 all. For example: supposing that in a colour panel with RGB sub-pixels a 
particular R sub-pixel is defective such that the colour point of that pixel is 
incorrect, then according to this embodiment of the method an artificial G- and 
B- defective sub-pixel are introduced such that the colour point or colour 
co-ordinates of the defective pixel remains correct as much as possible (but 

30 the luminance value is not correct). It is to be noted that it is not always 
possible to correct the colour point completely with the remaining sub-pixels. 
To restate this method: the drive values of the two remaining non-defective 
sub-pixels will be changed so that the colour point of the pixel as a unit 
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remains as close to the correct value as possible. It will be obvious for those 
skilled in the art that this is easy to perform once the (Y,x,y) coordinates of 
each sub-pixel type (for example red/green and blue sub-pixels in case of a 
colour display as in Fig. 2a) are available. These (Y,x,y) co-ordinates, where Y 
5 is the intensity and x,y are the chromaticity co-ordinates, can be measured 
easily for each of the sub-pixel types and at one or more drive levels. The 
masking pixels are then calculated with the normal minimisation problem for 
each colour independently where the artificial defects are treated as real 
defects. 

10 It is known that the human eye is more sensitive to intensity differences 

than to chromaticity differences. Therefore a third method allows a colour point 
error to keep the intensity error due to the defect as small as possible. This can 
be achieved by only or mainly minimising the intensity response of the eye. In 
this case the drive signals for driving the remaining non-defective sub-pixels 

15 will be changed in such a way that the luminance intensity error of the pixel as 
a unit is as small as possible, while the colour of the pixel as a unit may 
deviate from the colour originally intended to be displayed. This is again easy 
to perform once the (Y,x,y) co-ordinates of each sub-pixel type (for example 
red, green and blue sub-pixels in case of a colour display as in Fig. 2a) are 

20 available. This means that also in this case virtual defects will be introduced 
possibly making the chromaticity error larger but minimising the intensity error. 
It is for example known that red and blue sub-pixels have a smaller intensity 
value than a green sub-pixel at a same level of a drive signal. If a green sub- 
pixel is defective, the red and blue sub-pixels will be driven, according to the 

25 present embodiment of the present invention, so as to have a higher intensity 
level. 

Of course, it is also possible to mix the three methods described above. 
This can be favourable for instance if the goal would be to limit at the same 
time both the intensity and colour temperature errors with one of them possibly 
30 being more important than the other. 

It is to be noted that typically the PSF is (slightly) wavelength 
dependent. So different PSFs can be used for each sub-pixel colour. Fig. 7a 
shows a real green defective sub-pixel 71 present in the display 70. Fig. 7b 



WO 2005/052902 



PCT/EP2004/0 13436 



23 



shows the same green defective sub-pixel 70 and artificial red and blue 
defective sub-pixels 72, 73 introduced to retain the correct colour co-ordinate 
of the pixel The artificial defective "pixels 72, 73 are not really present in the 
display but are introduced by altering the driving level of these pixels. For the 
situation in Fig. 7b, the minimisation problem will be solved based on three 
defective sub-pixels: one really defective sub-pixel 71 and two artificially 
introduced defective sub-pixels 72, 73. 

The PSF of a diffraction limited optical system is given by (in polar co- 
ordinates): 



1 0 PSF(r') = 



r'= r 



where J1 is the Bessel function of the first kind and r" is given by 

Xf 

where D is the aperture diameter, f is the focal length and X is the wavelength 
of the light. This means that the exact PSF is dependent on the iris diameter of 

15 the eye. Therefore, an improvement could be to adapt the PSF used for the 
calculation based on the average luminance value of the display or some part 
of the display such as the neighbourhood of the defect and/or the average 
luminance value of the environment. 

In this way, the method does not only allow to take into account the 

20 information about the position of the human vision system with respect to the 
display and the display defects, such as e.g. the distance to the display or the 
viewing angle, but it also allows to take into account the environmental stray 
light intensity. 

To simplify the calculation, some changes to the algorithm can be 

25 made. 

A first possible change is to restrict the integration in Eq. 1 to a limited 
area around the defect. This is possible because the result of the costfunction 
(and the value of the PSF) typically decreases very fast with increasing 
distance from the defect. If symmetric PSFs are used or if the pixel structure is 
30 symmetrical, then it is often possible to apply some boundary conditions to the 
correction values of the masking pixels. For example: in case of a point- 
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symmetric PSF and a point symmetric pixel structure it is obvious that the 

required correction values for the masking pixels will show point symmetry 
also. _ - -- -- 

Another possible change can be to approximate the integration over a 
5 certain area as a summation over particular points in that area. This is 
generally used in mathematics. If calculation time is very important, then the 
two-dimensional minimisation problem can be transformed or approximated 
into a one-dimensional problem (by transforming or approximating the 
PSF(x',y') by PSF(O). 

10 Visual masking of the defect according to the present invention can be 

done both in software and in hardware. The correction transforms the image 
into a pre-corrected image based on any of the correction schemes of the 
present invention, as described above. Some possible implementations of 
where the correction can be done are shown in Fig. 8, which illustrates 

15 possible locations for a real-time correction system. As illustrated by (1), the 
pixel correction may be done by the CPU of the host computer, for instance in 
the driver code of the graphical card or with a specific application or embedded 
in a viewing application. Alternatively, as illustrated by (2) and (3), the pixel 
correction may be done in the graphical card, either in hardware or in firmware. 

20 According to still another embodiment, as illustrated by (4) and (5), pixel 
correction may be done in the display, either in hardware or in firmware. And 
according to yet another embodiment, as illustrated by (6), pixel correction may 
be done on the signal transmitted between the graphical card and the display, 
anywhere in the datapath. 

25 It is to be noted that thai a correction algorithm according to 

embodiments of the present invention can be executed both in real-time (at 
least at the frame rate of the display) or off-line (once, at specific times or at a 
frame rate lower than the display frame rate). 

The present invention has two main applications: 1) avoiding that a user 

30 of the display mistakes the defective pixel for a real signal present in the 
displayed image; which especially in case of radiology for example could make 
a radiologist treat the defect as really present and this could be a possible 
threat for quality of the diagnosis; and 2) avoiding frustration of the user 
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because his/her possibly new display shows one or more extremely visible 
pixel defects. 

A device according to the present invention comprises a vision 
measurement system, a set-up for automated, electronic vision of the 
5 individual pixels of the matrix addressed display, i.e. for measuring the light 
output, e.g. luminance, emitted or reflected (depending on the type of display) 
by individual pixels 14. The vision measurement system comprises an image 
capturing device, such as for example a flat bed scanner or a high resolution 
CCD camera, and possibly a movement device for moving the image capturing 

10 device and the display 12 with respect to each other. The image capturing 
device generates an output file, which is an electronic image file giving a 
detailed picture of the pixels 14 of the complete electronic display 12. Once an 
image of the pixels 14 of the display 12 has been obtained, a process is run to 
extract pixel characterisation data from the electronic image obtained from the 

1 5 image capturing device. 

Instead of luminance, also colour can be measured. The vision set-up is 
then slightly different, and comprises a colour measurement device, such as a 
colorimetric camera or a scanning spectrograph for example. The underlying 
principle, however, is the same: a location of the pixel and its colour are 

20 determined. 



